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ABSTRACT 


Traditional  analysis  and  prediction  of  political  policy  relies  principally  on  expert 
judgment.  Augmenting  expert  judgment  with  analytical  models,  based  upon  Game  Theory, 
purportedly  enhances  the  accuracy  of  predictions.  The  specific  claims  and  methodology  of  Dr. 
Bruce  Bueno  de  Mesquita  offer  an  example  of  such  analysis  for  review  and  evaluation.  This 
review  explains  his  methodology,  investigates  his  claims  of  prediction  accuracy,  evaluates  his 
ability  to  mitigate  analytical  biases,  and  assesses  two  of  his  2009  predictions  for  the  Islamic 
Republic  of  Iran.  Overall,  the  approach  does  not  reduce  the  amount  of  synthesis  or  judgment 
required  from  the  analyst.  It  shifts  it  away  from  the  prediction  itself  and  into  the  model’s 
construction.  The  method  offers  a  superior  capability  to  predict  discontinuous  events,  mitigates 
anchoring  bias,  and  enables  the  rapid  evaluation  of  alternatives.  However,  its  use  must  be 
restricted  to  problems  wherein  the  participants  can  freely  negotiate  the  outcome  to  the  limits  of 
their  power.  Bueno  de  Mesquita’ s  model  provides  generally  accurate  predictions.  Moreover, 
given  the  same  information  that  this  model  requires  a  traditional  analyst  would  have  reached  the 
same  predictions.  Ultimately,  Game  Theory  models  provide  an  effective,  but  not  revolutionary, 
method  for  structuring  analysis  that  can  enhance  the  prediction  of  discontinuous  events  in  certain 


contexts. 


Introduction 


Traditional  analysis  and  prediction  of  political  policy  relies  principally  on  expert 
judgment.  Game  Theory  advocates  claim  that  augmenting  expert  judgment  with  analytical 
models,  enhances  the  accuracy  of  predictions.  For  example,  Dr.  Bruce  Bueno  de  Mesquita,  a 
New  York  University  Political  Science  professor  and  consultant,  extolls  this  approach.  In  his 
book  The  Predictioneer’s  Game  and  a  February  2009  interview  with  TED  Talks,  he  shared  some 
insight  into  his  methods  for  predictive  analysis  and  made  a  few  predictions  about  the  future  of 
the  Islamic  Republic  of  Iran.  A  holistic  evaluation  Bueno  de  Mesquita’s  methods  and 
predictions,  requires  explanation  or  scrutiny  of  four  areas:  a  functional  explanation  of  the  model 
itself;  an  investigation  of  his  claims  of  accuracy;  an  assessment  of  the  model  as  an  analytical 
tool;  and  an  evaluation  of  the  veracity  of  his  predictions  about  Iran. 

Explanation  of  Method 

In  his  book,  Bueno  de  Mesquita  provides  a  good  deal  of  insight  into  his  methodology, 
including  the  origins  of  his  approach.  The  core  of  his  model  is  a  system  called  Policon,  which 
his  firm  implemented  for  the  Central  Intelligence  Agency  (CIA)  in  the  mid-1980s. 1  Policon  is  a 
fairly  straightforward  implementation  of  Game  Theory.  The  following  sections  explain  the 
fundamentals  of  Policon  and  provide  an  explanation  and  example  of  its  use. 

The  foundational  assumptions  of  the  Policon  model  are  that  people  know  what  they  want 
and  they  will  use  their  resources  to  obtain  it.  With  that  set  of  assumptions,  the  algorithm  requires 
only  three  input  variables.  They  are:  each  actor’s  preferred  outcome,  their  relative  power 
(Power),  and  how  important  the  issue  is  to  them  (Salience).  Then,  the  model  calculates  a  value 
for  how  much  energy  each  actor  would  apply,  for  or  against,  each  potential  outcome  (Utility). 


Finally,  the  algorithm  calculates  how  much  each  actor  would  support  every  outcome  under 
consideration,  as  an  isolated  instance. 

Support  =  (Power)  X  (Salience)  X  (Utility) 

This  equates  to  each  actor  asking  himself,  “if  outcome  ‘A’  was  the  only  thing  on  the 
table,  how  much  effort  would  I  put  towards  obtaining  it?”  and  doing  so  for  every  outcome  under 
consideration.  The  model  predicts  the  results  by  totaling  and  comparing  the  amount  of  support 
each  potential  outcome  received  from  all  actors.  The  outcome  with  the  most  support  will  win." 
A  more  robust  model  will  also  apply  an  assortment  of  mathematical  functions  to  address 
uncertainty  and  produce  statistical  results. 

That  basic  explanation  of  Policon  may  make  the  application  of  Game  Theory  appear  to 
be  a  fairly  straightforward  process  of  gathering  data  and  calculating  results.  But,  closer 
examination  reveals  that  a  tremendous  amount  of  judgment  and  art  underpin  the  model.  That 
judgment  manifests  itself  in  the  construction  of  the  range  of  possible  outcomes  and  the 
assignment  of  preferred  outcomes  to  each  actor. 

Game  Theory  proponents  frequently  claim  that  the  model  requires  a  single  assumption, 
that  the  actors  be  rational,  i.e.  can  linearly  order  their  preferences  for  things  and  act  accordingly. 
However,  this  is  only  one  half  of  a  pair  of  key  assumptions.  Its  partner  is,  that  the  analyst  can 
construct  a  linear  range  of  outcomes  that  effectively  captures  the  actors’  preferences  and  their 
order.4  The  actors  must  be  rational  and  the  model  must  include  rational  outcomes  for  all  actors. 

In  his  book,  Bueno  de  Mesquita  cites  an  example  of  a  seminar  for  environmentalists 
where  the  attendants  were  predicting  the  results  of  greenhouse  gas  emissions  policy  negotiations. 
The  students  proposed  only  modeling  in  the  range  between  the  status  quo  and  tougher 
restrictions.  But,  this  would  have  excluded  actors  who  actually  prefer  greater  greenhouse  gas 


emissions.  For  instance,  energy  and  manufacturing  companies  might  reasonably  find  it 
beneficial  to  their  bottom  line  to  produce  more  greenhouse  gases.5  The  students’  range  would 
have  failed  to  consider  their  real  preferred  outcomes.  This  would  skew  the  utility  factor  portion 
of  the  calculation  and  erroneously  assign  their  support  to  lower-end  regulations,  rather  than 
modeling  support  for  de-regulation.  So,  the  assumption  that  the  companies  are  rational  would  be 
true.  But,  the  model  would  have  predicted  irrational  behavior,  i.e.  the  companies  not  pursuing 
their  preferred  outcome.6 

In  order  to  numerically  capture  a  range  of  social  and  political  outcomes,  analysts  rely  on 
the  Spatial  Theory  of  Voting  (STV). 7  The  STV  asserts  that  locations  in  a  coordinate  system  can 
be  used  to  represent  relatively  how  “far  apart”  political  positions  are  from  one  another.  A 
common  two-axis  example  of  this  is  the  “Nolan”  chart,  Figure  1,  created  by  Libertarian  party 

o 

founder  David  F.  Nolan.  The  chart  uses  two  axes,  Personal  Freedom  and  Economic  Freedom 
to  form  four  ideological  zones.  The  greater  the  distance  between  two  positions  on  the  chart,  the 
greater  the  divergence  in  political  views  on  economic  and  personal  freedom. 


Figure  1.  Nolan  Chart 


While  the  Nolan  chart  exemplifies  STV,  a  two  axis  system  squares  the  number  of 
calculations  required  for  a  simulation.  In  order  to  conduct  manageable  calculations  with  10s  to 
100s  of  actors,  with  wide  ranges  of  possible  outcomes,  Policon  requires  a  single  axis.  Some 
analysts  may  reflexively  argue  that  politics  are  multidimensional  and  that  this  cannot  be  done. 
Despite  the  truth  of  the  first  assertion,  analysts  can  assess  specific  issues  by  themselves. 

Limiting  the  scope  of  the  analysis  to  a  single  question  usually  reveals  that  issue  to  which  the 
actors  consider  most  sensitive  to,  e.g.  distribution  of  control,  cost,  return  on  investment,  short- 
vs.  long-term  benefit,  etc.  Within  that  context,  the  CIA  analysts  who  worked  with  Policon  found 
it  manageable.9 

Figure  2  provides  an  example  of  a  linear  range  of  outcomes  for  a  complex  issue  that  is 
appropriate  to  this  type  of  modeling,  “where  should  the  oil  money  go?”  The  dominant  issue 
differentiating  preference  was  assessed  as  individual  versus  collective  benefit.  On  the  far  left 
side,  position  100,  is  the  funding  of  the  state’s  operations  with  all  oil  money.  This  represents 
both  complete  state  control  of  the  funding  and  the  smallest  direct  benefit  to  individual  citizens. 
As  the  scale  moves  to  the  right,  towards  position  0,  the  direct  benefits  of  oil  funding  move 
through  a  zone  of  social  welfare  activities  until  finally  reaching  direct  distribution  to  individuals 
with  no  government  oversight. 


Where  Should  the  Oil  Money  Go? 
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Figure  2.  Oil  Money  Usage  Options 


Now,  with  a  linear  scale  established,  the  “real”  analysis  can  begin.  The  analyst  must 
assign  a  numerical  position  to  each  group  being  modeled.  Obviously,  those  assignments  are 
subjective.  Making  them  requires  traditional  analytical  technique,  historical  knowledge,  and 
judgment.  To  tie  this  back  to  the  Nolan  chart,  a  strong  Statist  would  favor  position  100. 
Conversely,  a  Libertarian  would  favor  position  Zero.  The  Socialist  and  Conservative  positions 
lie  in  between  these  two  extremes.  For  this  example,  a  strong  Socialist  would  favor  a  position 
around  30,  while  a  Conservative  would  favor  a  position  near  50.  Note  that,  even  in  this 
relatively  benign  example,  the  placement  of  the  Socialists  and  Conservatives  can  generate 
considerable  argument.  Accurate  predictions  require  accurate  assignments  of  preferred 
outcomes. 

With  actors  identified,  the  allocation  of  their  preferred  outcomes  also  generates  an 
expected  utility  value  for  each  possible  outcome. 10  In  layman’s  terms,  expected  utility  answers 
the  question,  “how  valuable  do  I  consider  this  outcome?”  The  analyst  assigns  a  value  to  each 
potential  outcome,  based  upon  its  displacement  from  the  actor’s  preferred  outcome.  Each  actor’s 
preferred  outcome  is  assigned  a  utility  of  1 .0.  They  get  what  they  want.  From  that  point,  utility 
diminishes  with  distance  from  their  preferred  outcome.  While  not  required,  some  analysts  prefer 


to  allow  utility  values  to  go  negative  to  represent  outcomes  that  actors  oppose. 1 1  This  correlates 
with  the  social-survey  scale  of:  Strongly  Agree,  Agree,  Somewhat  Agree,  Somewhat  Disagree, 
Disagree,  and  Strongly  Disagree.  So,  a  simple  model  for  utility  on  a  100-point  scale  could  be 
expressed  as  follows: 

A ’s  Utility  for  Outcome  N  =  l-(Absolute  Value  (A ’s  Preferred  Outcome  -  Outcome  N)*  0.1) 

Therefore  actor  ‘A’  will  have  100%  utility  for  his  or  her  preferred  outcome,  50%  utility 
for  an  outcome  25  points  away,  zero  utility  for  an  outcome  50  points  away,  and  negative  utility, 
i.e.  oppose,  outcomes  greater  than  50  points  away. 

Now  that  the  analyst  has  defined  the  model  and  its  inputs,  the  calculations  can  begin. 

The  top  sub-table,  in  Figure  3,  captures  the  preferred  outcome  of  the  previously  discussed 
political  groups,  an  assigned  Power  rating,  and  an  assigned  Salience  value.  The  second  sub-table 
applies  the  example  formula  to  determine  each  group’s  Utility  for  each  outcome.  The  third  sub¬ 
tables  calculates  each  group’s  support  for  every  outcome  by  multiplying  their  Power,  Salience, 
and  Utility.  The  bottom  row  totals  the  support  from  every  group  for  each  outcome.  Finally,  the 
analyst  observes  the  outcome  with  the  most  support  and  translates  the  prediction  back  into 
words.  In  this  case,  outcome  position  50  scored  the  highest  with  75.4  points  of  support. 
Therefore,  the  model  predicts  that  the  negotiations  will  result  in  oil  money  being  directed 
towards  ‘Low  Income  and  Unemployment  Benefits.’ 


Figure  3.  Composite  Example  Results 

Given  the  small  number  of  actors  that  were  considered  and  the  distribution  of  power, 
simple  intuition  would  likely  have  predicted  the  same  result.  So,  what  did  the  model  really  do? 
First,  it  required  the  analyst  to  formally  structure  his  or  her  conceptualization  of  the  issue  itself. 
Second,  the  process  produces  a  traceable  and  editable  accounting  of  the  analyst’s  assessments  of 
Power,  Salience,  and  Preferred  Outcome.  Finally,  it  lays  a  foundation  to  model  much  larger 
problems,  whose  outcome  can  vary  with  time,  and  apply  statistical  tools  to  address  uncertainty. 


An  informed  analyst’s  intuition  would  be  very  hard  pressed  to  evaluate  the  positions  of  the  300- 
500  members  of  a  typical  parliament,  consider  shifts  in  Salience  over  time,  or  provide  short-, 
mid-,  and  long-range  predictions  on  the  same  issue. 

To  make  his  predictions  about  Iran,  Bueno  de  Mesquita’s  used  a  more  complex  model 
than  the  example.  In  addition  to  the  features  previously  described,  his  model  considered  each 
actor’s  commitment  to  their  position  and  calculated  their  ability  to  influence  each  other  over 
time.  However,  he  withheld  the  specific  inner-workings  of  those  models  in  his  lecture  and 
books.  Presumably,  those  processes  are  proprietary,  i.e.  essential  to  the  marketing  of  his  product. 
He  is  a  consultant  as  well  as  a  professor  and  operates  a  for-profit  business.  Never  the  less,  the 
information  available  allows  an  assessment  of  Bueno  de  Mesquita’s  predictions  and  claims  about 
the  model. 

Predictions  and  Claims 

During  his  TED  Talks  presentation,  he  made  reasonably  specific  predictions  about  the 
future  of  Iran’s  nuclear  program  and  President  Mahmoud  Ahmadinejad’s  power-base.  He  also 
stated  that  he  was  going  to  assess  the  future  of  the  theocracy  in  Iran,  but  did  not  actually  cover 
that  topic.  More’s  the  pity,  as  that  issue  holds  great  importance  for  the  future  of  Iran  and  its 
relations  with  its  neighbors.  But,  the  boldest  claim  that  he  made  was  that  the  Central  Intelligence 
Agency  (CIA)  says  that  his  predictions  have  been  90%  accurate.  All  of  these  statements,  as  well 
as  his  method  for  making  these  predictions  bear  review  and  assessment. 

In  his  presentation  and  in  his  books,  Bueno  de  Mesquita  cites  a  CIA  assessment  that  his 
predictions  were  90-percent  accurate  and  makes  a  corollary  claim  that  his  method  “hit  the  bulls- 
eye”  twice  as  often  as  traditional  analysis.  He  challenges  his  audience  to  look  it  up  and  even 


provides  the  reference.  This  review  accepted  his  challenge.  The  results  were  unimpressive  on 
all  counts. 

The  citation  is  not,  in  fact,  an  official  CIA  assessment  of  the  performance  of  his 
methodology.  It  is  an  article  published  in  an  internal  trade  journal  for  analysts  called  Studies  in 
Intelligence.  The  article,  by  Stanley  A.  Feder,  entitled  Factions  and  Policon:  New  Ways  to 
Analyze  Politics,  examines  the  use  of  a  pair  of  tools  that  political  analysts  used  in  the  mid- 
1980s.  14  Given  the  source  and  nature  of  the  document,  it  is  inaccurate  for  Bueno  de  Mesquita  to 
claim  a  CIA  endorsement  for  his  work.  Aggrandizing  the  source  of  his  citations  undennines  the 
credibility  of  the  rest  of  his  claims. 

In  the  same  vein,  Bueno  de  Mesquita  would  also  have  his  audiences  believe  that  a  90- 
percent  prediction  accuracy  rate  resulted  from  the  application  of  his  methods,  in  the  alleged  CIA 
assessment.  This  was  also  not  the  case.  The  actual  statement  is  provided  here,  in  its  proper 
context.  “Interestingly,  forecast  done  with  traditional  methods  and  Policon  were  found  to  be 
accurate  90  percent  of  the  time.  Thus,  while  traditional  methods  and  Policon- based  analysis  both 
scored  well  in  terms  of  forecast  accuracy,  Policon  offered  greater  detail  and  less  vagueness.”15 
Feder  was  actually  commenting  on  the,  unusually  high,  accuracy  of  all  of  the  political  analysis 
that  was  conducted  during  the  period.  He  was  not  attributing  a  90-percent  accuracy  rate  to 
Policon  as  an  endorsement  of  its  performance.  If  Feder  had  only  found  a  25-percent  accuracy 
rate  for  all  predictions,  in  all  probability,  Bueno  de  Mesquita  would  not  be  citing  his  work. 

The  preceding  quote  notes  that  Feder  also  made  an  assessment  of  the  specificity  of  the 
predictions.  In  The  Predictioneer’s  Game,  Bueno  de  Mesquita  states  that,  “A  declassified  CIA 
study  reports  that  my  forecasting  model  has  hit  the  bulls-eye  about  twice  as  often  as  the 
government’s  experts  who  provided  me  with  data.”16  The  study  technically  made  that  assertion. 


But,  it  has  been  removed  from  its  context.  As  a  result,  readers  would  assume  that  this  refers  to 
the  accuracy,  rather  than  the  specificity,  of  his  predictions.  In  order  to  provide  an  understandable 
comparison  of  specificity,  Feder  contrived  a  model  with  four  concentric  rings.  At  the  center 
were  ‘Highly  Detailed’  predictions.  As  the  rings  expanded  outward,  specificity  falls  off  from 
‘Moderately  Specific,  to  ‘General,’  and  finally  ‘Vague.’  Feder  actually  said  that,  “Both 
traditional  approaches  and  Policon  often  hit  the  target,  but  Policon  analyses  got  the  bulls-eye 
twice  as  often.”17  The  bulls-eye,  in  question  was  a  ‘Highly  Detailed’  prediction,  not  an  accurate 
prediction,  as  Bueno  de  Mesquita  would  lead  his  readers  to  believe. 

Clearly,  Bueno  de  Mesquita  is  more  interested  in  generating  excitement  about  the 
possibilities  of  Predictioneering,  than  the  accuracy  of  his  assertions.  All  of  his  books, 
presentations,  and  websites  use  these  misleading  claims.  These  claims  seriously  undercut  his 
credibility  and  the  prospects  for  using  this  model.  Over-selling  a  product  virtually  ensures 
under-delivery. 

Assessment  as  an  Analytical  Tool 

However,  that  does  not  mean  that  the  model  cannot  enhance  analysis.  Using  any  model 
helps  analysts  structure  their  analysis  and  resist  cognitive  biases  and  analytical  traps.  The 
Policon  model  offers  analysts  specific  advantages  in  combating  continuity  and  anchoring  biases. 
Additionally,  the  computational  model  enables  analysts  to  quickly  and  impartially  consider 
alternatives. 18  Finally,  the  structure  of  the  model  creates  a  matrix  for  identifying  and  challenging 
assumptions  about  each  actor’s  power,  preferred  outcome,  and  salience. 

Continuity  Bias  occurs  when  the  assumption  that  past  trends  will  continue  into  the  future 
goes  untested  and  is  taken  as  a  fact.  The  Policon  model  is  intrinsically  resistant  to  Continuity 
Bias  due  to  the  structure  of  the  algorithm  and  the  way  it  uses  historical  information.  Past  actions 


and  trends  merely  provide  the  starting  input  variables  for  each  actor.  The  model  does  not  have  a 
mechanism  for  representing  the  influence  of  memories  of  past  alliances,  actions,  or  events.  For 
example,  trend  analysis  might  look  at  two  factions  and  assert  that,  their  consistent  antagonism 
towards  each  other  will  lead  to  future  opposition.  Such  a  prediction  may  ultimately  prove  true. 
But,  trend  analysis  tends  to  react  very  slowly  to  change.  The  large  number  of  data  points  in  the 
past  tends  to  outweigh  present  factors.  Policon  simulations  test  each  possible  outcome 
independently  and  may  show  those  same  two  factions  supporting  the  same,  winning, 
compromise  outcome.  This  possibility  of  discontinuous  outcomes  mitigates  the  continuity  bias. 

Policon  also  offers  significant  resistance  to  Anchoring  Bias.  Anchoring  bias  results  from 
the  initial  formulation  of  a  context.  Initial  conditions  tend  to  dominate  predictions  developed  by 
modifying  an  initial  hypothesis  with  new  data. 19  In  other  words,  new  information  does  not 
influence  the  outcome  as  much  as  it  objectively  should.  Policon  does  not  require  the  analyst  to 
develop  a  hypothesis  for  the  prediction.  Instead,  the  analyst  sets  up  the  model  and  the 
conditions,  then  calculates  a  prediction  without  forming  a  hypothesis.  Consequently,  using  the 
model  mitigates  Anchoring  Bias  by  removing  the  mechanisms  that  facilitate  it.'  In  order  to 
inject  an  Anchoring  Bias,  an  analyst  would  have  to  revise  the  other  inputs  and  re-run  the  model 
to  uphold  an  original  position.  This  type  of  behavior  should  trigger  self-inspection. 

Policon  facilitates  the  evaluation  of  numerous  alternatives  in  two  different  ways.  First, 
the  model  itself  calculates  the  support  for  every  potential  outcome  within  its  range,  not  just  those 
favored  by  specific  groups.  This  should  be  compared  to  the  Analysis  of  Competing 
Hypotheses  (ACH)  method,  a  popular  method  in  predictive  intelligence  analysis.  Mechanically, 

it  tests  the  available  information,  against  a  set  of  hypotheses.  Only  the  amount  of  work  required 
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to  produce  each  assessment  limits  the  number  of  potential  outcomes  that  can  be  evaluated." 


Analysts  using  the  ACH  method  typically  assess  three  to  five  cases;  worst-case,  best-case,  and  a 
few  in  between.  Second,  the  Policon  model  also  allows  analysts  to  easily  and  impartially 
consider  numerous  “what  if’  scenarios.  Variables  can  be  adjusted,  in  a  controlled  manner,  and 
the  model  re-run  to  quickly  assess  the  impacts  of  variance  changes,  e.g.  actors  losing  power,  an 
issue  gaining  salience,  etc. 

Using  Policon  to  frame  a  line-of-analysis  can  help  make  assumptions  explicit.  If  the 
analytical  team  documents  their  rationale  for  assigning  values,  it  codifies  their  assumptions. 
Additional  rigor,  in  compliance  with  the  CIA’s  analysis  guide  for  Assumption  Checking ,  can  be 
achieved  by  two  means.  First,  a  series  of  comparisons  of  each  actor  can  be  made,  relative  to 
each  other,  to  confirm  the  team’s  opinion  on  the  relative  power  and  salience  rankings,  i.e.  is  actor 
‘A’  really  20%  more  powerful  than  actor  ‘B’  on  this  issue?  Why  do  the  analysts  think  that? 
Second,  the  model  allows  for  sensitivity  analysis  of  the  predictions,  which  may  be  instrumental 
in  identifying  the  critical  actors  to  change  the  outcome.  From  a  political  intelligence  perspective, 
the  process  of  conducting  the  rack-and-stack  itself  may  be  more  valuable  than  the  model’s 
prediction. 

Naturally,  using  a  model  to  aid  analysis  is  not  a  panacea.  Before  using  any  model,  an 
analyst  must  consider  a  series  of  questions.  First,  does  the  model  correlate  well  with  the  system 
being  modeled?  Second,  is  there  sufficient  validated  information  to  feed  into  the  model?  Third, 
does  the  model  itself  create  new  biases  or  cognitive  traps?  All  three  of  these  risks  affect  the 
Policon  model. 

The  Policon  algorithm  is  fundamentally  based  on  the  idea  that  the  position  with  the  most 
support  will  be  the  outcome.  This  correlates  well  with  a  hypothetical  multi-party  negotiation. 
But,  political  positions  are  not  always  freely  negotiated.  For  instance,  the  Supreme  Ayatollah  of 


Iran  has  ultimate  control  of  foreign  policy.  Even  a  majority  of  the  political  power  blocs  in  Iran 
supporting  a  position  does  not  ensure  that  the  Ayatollah  will  pursue  it.  In  his  work  on  North 
Korea,  Bueno  de  Mesquita  admits  as  much.  The  model  must  be  applied  to  systems  where  free 
negotiations  occur.  For  example,  an  internal  issue  that  has  been  delegated  to  the  Iranian  Majlis 
to  resolve  may  be  a  good  candidate  for  the  model.  Presumably,  a  delegated  issue  will  not  draw  a 
veto  from  the  Supreme  Ayatollah. 

All  models  require  good  data.  But,  the  truism  of,  “garbage  in,  garbage  out,”  applies 
acutely  to  Policon  because  it  requires  the  distillation  of  an  unusually  large  amount  of  information 
into  merely  three  input  variables.  The  vast  majority  of  the  work  and  judgment  goes  into 
constructing  the  model.  The  collection,  validation,  interpretation,  and  fusion  of  the  factors  that 
determine  an  actor’s  preferred  outcome,  power,  and  salience,  constitutes  “doing  the  analysis”  for 
any  method. 

If  presented  with  intellectual  integrity,  the  Game  Theory  approach  does  offer  analytical 
benefits.  Policon  can  be  described  as  a  modeling  tool  that  rides  on  top  of  an  analytical  effort. 
However,  Bueno  de  Mesquita,  prefers  the  wholesale  outsourcing  of  responsibility  for  the  inputs. 
Regarding  this  laborious  process,  he  argues  that, 

“Luckily,  there  is  a  more  efficient  way  to  get  the  information — ask  the  experts.  It 
is  that  simple.  Experts  have  invested  years  in  learning  a  place’s  culture,  language, 
and  history.  They  follow  the  intimate  political  details  that  go  on  in  the  area  they 
study.  If  anyone  knows  who  will  try  to  shape  decisions,  how  influential  those 
people  can  be,  where  they  stand,  and  how  much  they  care  about  an  issue,  it  is  the 
experts.  Come  to  think  of  it,  isn’t  knowing  this  information  what  it  means  to  be 
an  expert?”25 


While  no  one  would  fault  the  idea  of  consulting  with  experts  as  a  research  strategy,  his 
delivery  glosses  over  the  critical  synthesis  step,  where  the  analyst  translates  those  “expert 
opinions”  into  numerical  values.  Clearly,  practitioners  of  Predictioneering  must  take  more  care 
with  the  underlying  data  and  its  transformations  than  the  marketing  department  does  with  its 
messaging. 

Presupposing  that  the  analyst  has  selected  a  system  that  correlates  well  with  the  model, 
collected  and  validated  the  data,  then  carefully  transformed  his  or  her  understanding  into 
numerical  values,  Policon  faces  one  more  challenge  that  is  specific  to  this  form  of  modeling. 
Humans  understand  the  world  in  a  story-based  framework.  The  simple  human  preference  for  an 
explanation  to  make  a  “good  story”  results  in  Narrative  Bias. 

The  probability  of  a  Narrative  Bias  is  proportional  to  the  amount  of  interpretation 
required  for  a  process.  Policon  requires  a  lot  of  interpretation.  The  analyst  contrives  and  orders 
the  range  of  possible  outcomes.  He  or  she  also  sets  the  four  major  variables:  Power,  Preferred 
Outcome,  Salience,  and  Utility  equation.  Finally,  simply  explaining  the  outcome  requires  a 
degree  of  story-telling.  The  model  only  produces  a  spreadsheet  full  of  numbers,  which  the 
analyst  must  express  in  terms  of  political  outcomes.  People  generally  accept  a  prediction 
explained  in  narrative  form  much  more  easily  than  an  abstract  numerical  score. 

Bueno  de  Mesquita  gives  us  a  great  example  of  Narrative  Bias  in  his  explanation  of  the 
data  from  his  TED  Talks  interview.  “. .  .we  get  to  a  stable  equilibrium  outcome. .  .it's  probably  an 
equilibrium  that  the  United  States  can  live  with. .  .Iran  will  achieve  that  nationalist  pride  by 
making  enough  weapons-grade  fuel. .  .so  that  they  could  show  that  they  know  how  to  make 
weapons.”  In  a  20-second  explanation,  Bueno  de  Mesquita  has  injected,  by  way  of 
explanation,  a  significant  narrative  based  upon  his  own  value-judgments  that  did  not  derive  from 


the  model.  The  model  merely  found  an  equilibrium  point  indicating  that  there  was  more  support 
for  the  option  of  “produce  enough  weapons-grade  fuel  for  research”  than  any  other  option  in  the 
range.  The  explanation  provided  no  information  about  the  US’s  ability  to  accept  that  outcome  or 
about  the  importance  of  Iran’s  national  pride  in  relation  to  their  nuclear  program.  Bueno  de 
Mesquita  has  injected  his  own  storytelling  vision  to  build  the  audiences  confidence  in  his 
predictions. 

Ultimately,  Policon  analysts  should  bear  in  mind  that  Policon  is  a  tool,  like  any  other.  It 
helps  an  analyst  structure  information  and  evaluate  possible  outcomes.  But,  it  also  suffers  from 
the  usual  limitations  of  modeling  and  especially  from  the  quality  of  the  infonnation  that  goes 
into  it.  Analysts  should  not  expect  that  the  model  will  work  effectively  in  all  situations. 
Therefore,  they  must  carefully  select  the  problems  to  which  they  apply  it. 

Assessment  of  Predictions 

This  review  of  the  Policon  model  itself  and  Bueno  de  Mesquita’s  claims  of  accuracy  lead 
to  the  conclusion  that  Game  Theory  modeling  serves  only  as  another  tool  in  the  analyst’s  tool¬ 
box.  But,  his  TED  Talk  predictions  also  provide  an  opportunity  to  assess  his  implementation  of 
the  tool.  In  the  hands  of  a  strong  practitioner,  it  can  produce  good  results. 

In  his  TED  Talk,  Bueno  de  Mesquita  stated  that  he  would  assess  President 
Ahmadinejad’s  power-base.  While  he  does  not  provide  details  regarding  the  exact  method  of 
calculation,  he  clearly  employs  a  more  complex  version  of  Game  Theory  analysis  than  the 
Policon  model.  In  Figure  4,  extracted  from  his  presentation,  major  political  factions  are  shown 
gaining  and  losing  power  over  time. 
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Figure  4.  TED  Talks:  Political  Winners  and  Losers 

This  prediction  offers  two  important  take-aways.  First,  President  Ahmadinejad  has  much 
less  power  than  most  Americans  might  assume.  All  of  the  other  factions  portrayed  in  the  chart 
hold  more  power  than  he  does.  Also,  his  predicted  loss  of  power  primarily  benefits  the  moneyed 
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interests:  the  bankers,  the  oil  people,  and  the  bazaaries." 

The  most  visible  signals  of  Ahmadinejad’s  waning  power  occurred  beyond  the 
prediction’s  timeline.  In  May  of  2012,  his  faction  did  not  fare  well  in  the  Parliamentary  election, 
securing  43  of  290  seats.  The  largest  faction,  led  by  Ali  Larijani,  one  of  Khamenei’s  allies,  took 
98  seats.  This  shift  in  power  in  the  Majlis  made  his  last  two  years  in  office  much  more  difficult 
politically.  Prior  to  the  elections,  the  Majlis  had  already  taken  action  to  undermine  him.  The 
Majlis  stripped  the  Presidency  of  oversight  authority  of  state-owned  energy  assets  and  hanks. 
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The  Majlis  also  initiated  legal  proceedings  against  several  of  his  appointees.  Additionally, 
over  the  last  two  years  of  his  Presidency,  the  Iranian  press  reported  several  direct  confrontations 
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between  Khamenei  and  Ahmadinejad." 

This  predicted  loss  of  power  also  played  out  in  the  2013  elections,  wherein  Hasan 
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Rouhani  defeated  Ahmadinejad’s  favored  successor.  After  Ahmadinejad  stepped  down  from 


the  Presidency,  Khamenei  appointed  him  to  the  Expediency  Council,  a  body  designed  to  mediate 
disputes  between  the  Majlis,  or  parliament,  and  the  President.  Ahmadinejad  will  have  to 
recognize  the  leadership  of  the  head  of  this  council,  ex -president  Akbar  Hashemi  Rafsanjani, 
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who  has  regularly  criticized  Ahmadinejad’s  political  and  economic  policies  in  the  past. 
Ahmadinejad  has  little  prospect  of  wielding  significant  influence  in  that  position. 

Ahmadinejad  also  appears  to  have  recognized  his  declining  political  fortunes  and  made 
provisions  for  a  less  prominent  role.  In  fact,  it  appears  that  he  created  a  “landing  pad”  for 
himself  and  his  supporters,  prior  to  leaving  office.  He  applied  for  permission  to  establish  a 
University  in  Tehran  in  2010.  He  then  proceeded  to  staff  it  with  co-workers  and  supporters 
including:  Shamsaddin  Hosseini,  ex-minister  of  economy  and  finance;  Hamid  Baghaei,  ex -head 
of  the  Iranian  presidential  administration;  and  Behrouz  Moradi,  ex-vice  president  of  planning. 

As  one  of  his  last  acts  as  President  he  apparently  moved  $6.45  million  from  a  Presidential  bank 
account  to  a  university.  However,  later  reports  claimed  that  the  University  returned  the  money 
after  his  opponents  initiated  an  investigation. 32  All  of  this  supports  the  conclusion  that 
Ahmadinejad’s  power  has  waned  considerably. 

Bueno  de  Mesquita’s  also  predicted  the  end-point  of  the  Iranian  nuclear  program.  For 
this  prediction,  he  began  by  establishing  a  sequential  range  of  objectives  that  Iran  could  pursue. 
He  then  assessed  some  combination  of  support  and  risk  for  each  option  across  the  full  gamut  of 
stakeholders  including  the  United  States,  the  International  Atomic  Energy  Agency  (IAEA),  and  a 
host  of  domestic  power  blocks.  His  model  predicted  that,  over  a  two  year  period,  moderate 
elements  would  exercise  sufficient  influence  to  undermine  support  for  a  nuclear  weapons 
program  and  stabilize  on  a  civilian  program,  see  Figure  5. 33  This  would  maintain  Iranian 
prestige,  while  gradually  retreating  from  Ahmadinejad’s  bellicose  rhetoric. 
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Figure  5.  Iran's  Nuclear  Policy 

Monitoring  the  amount  of  uranium  that  Iran  has  enriched  to  20%  purity  provides  one  of 
the  few  empirical  indicators  of  the  status  and  direction  of  the  program.  The  Iranians  have 
continued  to  enhance  their  capability  and  capacity  to  refine  uranium.  However,  they  have 
slowed  their  actual  rate  of  production,  see  Figure  6.  Had  production  continued  at  the  same  rate 
as  in  the  fall  of  2012,  the  Iranians  would  have  breached  the  threshold  established  by  the  Israelis 
in  June  of  20 13. 34  Unfortunately,  this  action  does  not  necessarily  indicate  a  policy  shift.  Iran 
may  simply  not  want  to  provoke  an  international  reaction  until  they  possess  and  can  assemble  all 
of  the  elements  for  building  an  employable  weapon. 


Overall,  the  program  appears  to  be  functioning  consistently  with  Ahmadinejad’s 
successor’s  policies.  Iran  continues  to  buy  time  for  their  program,  e.g.  not  pushing  the  Israeli 
red-line,  while  maintaining  their  claims  of  only  pursuing  peaceful  purposes.  This  does  not 
necessarily  represent  a  change  in  objective.  All  policies  ultimately  originate  with  Ayatollah 
Khamenei.  Rouhani  does  not  have  the  authority  to  actually  change  Iran’s  nuclear  policy, 
regardless  of  how  much  dialogue  he  conducts.  Until  Khamenei’s  public  position  changes  on  the 
issue,  Iran  appears  content  to  stroll  towards  nuclear  capability,  at  least  publicly. 

In  hind  sight,  Bueno  de  Mesquita  failed  to  accurately  predict  Iran’s  nuclear  policy.  But, 
he  did  manage  to  predict  the  program’s  actions.  Even  though  Iran  did  not  alter  its  policies,  some 
of  the  forces  that  were  considered  in  the  analysis  did  impact  their  actions.  While  he  did  not  “hit 
the  bulls-eye,”  the  prediction  would  have  effectively  infonned  mid-range  decision  making.  That 
is  still  useful  predictive  analysis. 


Conclusion 


Ultimately,  Game  Theory  models,  such  as  Policon,  provide  analysts  with  effective  tools 
to  structure  their  analysis  and  offer  an  alternative  to  trend-based  methods.  The  method  offers 
superior  capability  to  predict  discontinuous  events,  mitigates  anchoring  bias,  and  enables  the 
rapid  evaluation  of  alternatives.  But,  its  use  must  be  restricted  to  problems  wherein  the 
participants  can  freely  negotiate  the  outcome  to  the  limits  of  their  power.  Despite  Bueno  de 
Mesquita’s  exaggerated  claims  of  accuracy,  he  demonstrated  the  ability  to  use  the  model 
effectively  in  his  predictions. 
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